Mesh network of nearby mobile devices as a combined speaker system for audio

ABSTRACT

Implementations disclose a mesh network of nearby mobile devices as a combined speaker system for audio. A method includes receiving a prompt to join a speaker mesh network as a master computing device of the speaker mesh network, the speaker mesh network comprising one or more other computing devices detected within a geographic vicinity of a computing device, generating, by the computing device acting as the master computing device, playback synchronization instructions for the one or more other computing devices, the playback synchronization instructions to cause playback of a media item to be synchronized at a playback timestamp relative to a clock time, and transmitting the generated playback synchronization instructions to the one or more other computing devices.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional PatentApplication No. 15/188,488, filed Jun. 21, 2016, entitled “MESH NETWORKOF NEARBY MOBILE DEVICES AS A COMBINED SPEAKER SYSTEM FOR AUDIO”, whichis incorporated herein by reference for all purposes.

TECHNICAL FIELD

This disclosure relates to the field of content sharing platforms and,in particular, to a mesh network of nearby mobile devices as a combinedspeaker system for audio.

BACKGROUND

Content-sharing platforms and their corresponding websites and mobileapplications (“apps”) allow users to play and watch music and videos.Users often find themselves in situations where they would like to playmusic or watch content, but do not have an established speaker setup toutilize. For example, a user may prefer to listen to music or watchcontent while at a picnic in the park, a BBQ in the back yard, and soon. Multi-speaker systems are advanced and provide high-quality audio,but many users have not invested in these types of systems. Furthermore,these types of systems are usually kept at home. For occasions outsideof the house, or even in different rooms of the house, videos are oftenwatched on a mobile device with a single mobile phone speaker that doesnot provide high-quality audio output.

SUMMARY

The following is a simplified summary of the disclosure in order toprovide a basic understanding of some aspects of the disclosure. Thissummary is not an extensive overview of the disclosure. It is intendedto neither identify key or critical elements of the disclosure, nordelineate any scope of the particular implementations of the disclosureor any scope of the claims. Its sole purpose is to present some conceptsof the disclosure in a simplified form as a prelude to the more detaileddescription that is presented later.

In an aspect of the disclosure, a method includes detecting one or moreother computing devices within a geographic vicinity of a computingdevice, transmitting a confirmation to join a speaker mesh network withthe detected one or more other computing devices, receiving a playbacksynchronization instruction for a media item, adjusting playback of themedia item according to the playback synchronization instruction,wherein the one or more other computing devices in the speaker meshnetwork provide synchronized playback of the media item according to theplayback synchronization instruction, receiving a stereo sound and audioequalization instruction, and applying, according to the stereo soundand audio equalization instruction, stereo sound and audio equalizationsettings to the playback of the media item at the computing device.

In one implementation, the playback synchronization instruction includesidentification of the media item, a timestamp in the media item, and asynchronized clock time. Furthermore, the stereo sound and audioequalization settings at the one or more other computing devices aredifferent than the stereo sound and audio equalization settings at thecomputing device. In addition, the method may further include presentinga prompt in a user interface (UI) of the computing device, the prompt toquery whether to join the speaker mesh network with the detected one ormore other computing devices, and receiving, via the UI having theprompt, the confirmation to join the speaker mesh network.

In another implementation, the method further includes transmitting oneor more sensor readings to a master computing device of the speaker meshnetwork. Furthermore, the one or more sensor readings may include anaudio input at one or more microphones of the computing device. Inaddition, the one or more sensor readings may include a compass readingof the computing device. In one implementation, the one or more sensorreadings comprise a global positing system (GPS) reading of thecomputing device. Additionally, the one or more sensor readings maycomprise a barometric sensor reading of the computing device.

In some implementations, the one or more sensor readings are used by themaster computing device to determine a topology of the speaker meshnetwork. Furthermore, the stereo sound and audio equalizationinstructions may be based on the topology of the speaker mesh network.In addition, the method may further include transmitting updated sensorreadings to the master computing device on at periodic time intervals,the updated sensor readings used by the master computing device toupdate the topology of the speaker mesh network and to send updatedstereo sound and audio equalization instructions. Additionally, thestereo setting may include proportions of audio tracks of the media itemto output during the playback of the media item, and wherein the audioequalization settings may include a proportion of bass and a proportionof treble to apply to the playback of the media item.

Computing devices for performing the operations of the above describedmethod and the various implementations described herein are disclosed.Computer-readable media that store instructions for performingoperations associated with the above described method and the variousimplementations described herein are also disclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings.

FIG. 1 is a block diagram illustrating an exemplary network architecturein which implementations of the disclosure may be implemented.

FIG. 2 is a block diagram of speaker mesh network server component andspeaker mesh network client component, in accordance with animplementation of the disclosure.

FIGS. 3A through 3C are exemplary screenshots of a UI for a mesh networkof computing devices as a combined speaker system for audio, accordingto implementations of the disclosure.

FIG. 4 is a flow diagram illustrating a method for a mesh network ofcomputing devices as a combined speaker system for audio according to animplementation.

FIG. 5 is a flow diagram illustrating another method for a mesh networkof computing devices as a combined speaker system for audio, accordingto an implementation.

FIG. 6 is a block diagram illustrating one implementation of a computersystem, according to an implementation.

DETAILED DESCRIPTION

Aspects and implementations of the disclosure are directed to a meshnetwork of nearby mobile devices as a combined speaker system for audio.In one implementation, multiple computing devices, such as mobiledevices including cell phones and tablet devices, can join together as amesh network speaker system. A mesh network refers to a network topologyin which each node relays data for the network. Each node in the meshnetwork cooperates in the distribution of data in the network. As such,in a speaker mesh network, each computing device (e.g., node) joined inthe mesh network cooperates to create improved audio output (e.g.,louder and/or higher quality audio output) of a media item.

In some implementations, speaker mesh network components in either, orboth of, client computing devices and/or server computing devices workin conjunction to provide the mesh network speaker system. In oneimplementation, when a speaker mesh network component in a clientcomputing device detects other nearby client computing devices, thespeaker mesh network component prompts the client computing device tojoin the speaker mesh network. In addition, speaker mesh networkcomponents in the other nearby client computing devices similarly promptthe nearby computing devices to join the speaker mesh network. Once thespeaker mesh network is established, a speaker mesh network component ata master computing device (i.e., one of the client computing devices ora server device) acts as a master in a topology of the speaker meshnetwork and coordinates playback synchronization, stereo sound, andaudio equalization signaling among the client computing devices of thespeaker mesh network.

The stereo sound and audio equalization signaling may be based on thetopology of the speaker mesh network as determined by the mastercomputing device. The master computing device may determine the topologybased on sensor readings sent to the master computing devices from theclient computing devices. In some implementations, the speaker meshnetwork component may also implement a feedback system to listen to theoverall audio output among the client computing devices of the speakermesh network and make adjustments to the stereo sound and audioequalization signaling to optimize audio output in the speaker meshnetwork.

Existing approaches to providing audio do not provide for improved audioquality by determining a topology of the network and providinginstructions for stereo sound and audio equalization signaling among thecomputing devices in the network. For example, existing approaches donot provide a master computing device that understands relativepositioning among nodes of the speaker mesh network and adjusts audiooutput of each computing device node based on such an understanding.Implementations of the disclosure provide improved audio quality usingmultiple computing devices by creating a mesh network of nearby mobiledevices as a combined speaker system for audio. In some implementations,audio output is synchronized among the devices, and stereo sound andaudio equalization are provided among the devices based on a topology ofthe speaker mesh network.

Implementations of the disclosure often reference videos for simplicityand brevity. However, the teaching of the present disclosure are appliedto media items generally and can be applied to various types of contentor media items, including for example, video, audio, text, images,program instructions, etc.

FIG. 1 illustrates an example system architecture 100, in accordancewith one implementation of the disclosure. The system architecture 100includes client devices 110A through 110Z, a network 105, a data store106, a content sharing platform 120, and a server 130. In oneimplementation, network 105 may include a public network (e.g., theInternet), a private network (e.g., a local area network (LAN) or widearea network (WAN)), a wired network (e.g., Ethernet network), awireless network (e.g., an 802.11 network or a Wi-Fi network), acellular network (e.g., a Long Term Evolution (LTE) network), routers,hubs, switches, server computers, and/or a combination thereof. In oneimplementation, the data store 106 may be a memory (e.g., random accessmemory), a cache, a drive (e.g., a hard drive), a flash drive, adatabase system, or another type of component or device capable ofstoring data. The data store 106 may also include multiple storagecomponents (e.g., multiple drives or multiple databases) that may alsospan multiple computing devices (e.g., multiple server computers).

The client devices 110A through 110Z may each include computing devicessuch as personal computers (PCs), laptops, mobile phones, smart phones,tablet computers, netbook computers, network-connected televisions, etc.In some implementations, client device 110A through 110Z may also bereferred to as “user devices.” Each client device includes a mediaviewer 111. In one implementation, the media viewers 111 may beapplications that allow users to view content, such as images, videos,web pages, documents, etc. For example, the media viewer 111 may be aweb browser that can access, retrieve, present, and/or navigate content(e.g., web pages such as Hyper Text Markup Language (HTML) pages,digital media items, etc.) served by a web server. The media viewer 111may render, display, and/or present the content (e.g., a web page, amedia viewer) to a user. The media viewer 111 may also display anembedded media player (e.g., a Flash® player or an HTML5 player) that isembedded in a web page (e.g., a web page that may provide informationabout a product sold by an online merchant). In another example, themedia viewer 111 may be a standalone application (e.g., a mobileapplication or app) that allows users to view digital media items (e.g.,digital videos, digital images, electronic books, etc.). According toaspects of the disclosure, the media viewer 111 may be a content sharingplatform application for users to record, edit, and/or upload contentfor sharing on the content sharing platform, and implements a meshnetwork of computing devices as a combined speaker system for audio.

The media viewers 111 may be provided to the client devices 110A through110Z by the server 130 and/or content sharing platform 120. For example,the media viewers 111 may be embedded media players that are embedded inweb pages provided by the content sharing platform 120. In anotherexample, the media viewers 111 may be applications that are downloadedfrom the server 130.

In general, functions described in one implementation as being performedby the content sharing platform 120 can also be performed on the clientdevices 110A through 110Z in other implementations, if appropriate. Inaddition, the functionality attributed to a particular component can beperformed by different or multiple components operating together. Thecontent sharing platform 120 can also be accessed as a service providedto other systems or devices through appropriate application programminginterfaces, and thus is not limited to use in websites.

In one implementation, the content sharing platform 120 may be one ormore computing devices (such as a rackmount server, a router computer, aserver computer, a personal computer, a mainframe computer, a laptopcomputer, a tablet computer, a desktop computer, etc.), data stores(e.g., hard disks, memories, databases), networks, software components,and/or hardware components that may be used to provide a user withaccess to media items and/or provide the media items to the user. Forexample, the content sharing platform 120 may allow a user to consume,upload, search for, approve of (“like”), dislike, and/or comment onmedia items. The content sharing platform 120 may also include a website(e.g., a webpage) or application back-end software that may be used toprovide a user with access to the media items.

In implementations of the disclosure, a “user” may be represented as asingle individual. However, other implementations of the disclosureencompass a “user” being an entity controlled by a set of users and/oran automated source. For example, a set of individual users federated asa community in a social network may be considered a “user”. In anotherexample, an automated consumer may be an automated ingestion pipeline,such as a topic channel, of the content sharing platform 120.

The content sharing platform 120 may host data content, such as mediaitems 121. The data content can be digital content chosen by a user,digital content made available by a user, digital content uploaded by auser, digital content chosen by a content provider, digital contentchosen by a broadcaster, etc. Examples of a media item 121 can include,and are not limited to, digital video, digital movies, digital photos,digital music, website content, social media updates, electronic books(ebooks), electronic magazines, digital newspapers, digital audio books,electronic journals, web blogs, real simple syndication (RSS) feeds,electronic comic books, software applications, etc. In someimplementations, media item 121 is also referred to as a content item.

A media item 121 may be consumed via the Internet and/or via a mobiledevice application. For brevity and simplicity, an online video (alsohereinafter referred to as a video) is used as an example of a mediaitem 121 throughout this document. As used herein, “media,” media item,”“online media item,” “digital media,” “digital media item,” “content,”and “content item” can include an electronic file that can be executedor loaded using software, firmware or hardware configured to present thedigital media item to an entity. In one implementation, the contentsharing platform 120 may store the media items 121 using the data store106.

In one implementation, the server 130 may be one or more computingdevices (e.g., a rackmount server, a server computer, etc.). In oneimplementation, the server 130 may be included in the content sharingplatform 120. The server 130 may include a speaker mesh network servercomponent 140. The speaker mesh network server component 140 enablesmultiple computing devices, such as mobile devices including phones andtablets, to join together in a mesh network speaker system inimplementations of the disclosure. A mesh network refers to a networktopology in which each node relays data for the network. Each node inthe mesh network cooperates in the distribution of data in the network.As such, in a speaker mesh network, each computing device (e.g., node)joined in the mesh network cooperates to create improved audio output(e.g., louder and/or higher quality audio output) of a media item.

In some implementations, client device 110A-110Z may include a speakermesh network client component 115 that works in conjunction with (or insome implementations, works independently of) speaker mesh networkserver component 140 to provide the mesh network speaker system. Speakermesh network client component 115 may perform implementations of thedisclosure independently of speaker mesh network server component 140 ofserver 130, or may work in conjunction with speaker mesh network servercomponent 140. Although the following description may refer to speakermesh network server component 140 performing implementations of thedisclosure, it should be understood that the functionality of speakermesh network server component 140 may be similarly performed solely by,and/or in conjunction with, speaker mesh network client component 115 atclient device 110A-110Z.

In one implementation, speaker mesh network client component 115 of aclient device 110A-110Z, may detect other nearby client devices110A-110Z and prompt the other devices 110A-110Z to join the speakermesh network. Once the speaker mesh network is established, speaker meshnetwork server component 140 may act as a master computing device in thespeaker mesh network topology and coordinate playback synchronization,stereo sound, and audio equalization signaling among the nodes (e.g.,client devices 110A-110Z) of the speaker mesh network. In someimplementations, speaker mesh network server component 140 may implementa feedback system to listen to the overall audio output among thespeaker mesh network nodes and make adjustments to stereo sound andaudio equalization signaling to optimize audio output in the speakermesh network. As discussed above, in some implementations, speaker meshnetwork client component 115 may act as the master computing device forthe speaker mesh network (e.g., when an Internet connection is notavailable).

In some implementations, speaker mesh network server component 140 ofserver 130 may interact with content sharing platform 120 to provideimplementations of the disclosure. Further description of the speakermesh network server component 140 and the speaker mesh network clientcomponent 115, as well as their specific functions, are described inmore detail below with respect to FIG. 2.

Although implementations of the disclosure are discussed in terms ofcontent sharing platforms and promoting social network sharing of acontent item on the content sharing platform, implementations may alsobe generally applied to any type of social network providing connectionsbetween users. Implementations of the disclosure are not limited tocontent sharing platforms that provide channel subscriptions to users.

In situations in which the systems discussed here collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity to control whether the contentsharing platform 120 collects user information (e.g., information abouta user's social network, social actions or activities, profession, auser's preferences, or a user's current location), or to control whetherand/or how to receive content from the content server that may be morerelevant to the user. In addition, certain data may be treated in one ormore ways before it is stored or used, so that personally identifiableinformation is removed. For example, a user's identity may be treated sothat no personally identifiable information can be determined for theuser, or a user's geographic location may be generalized where locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular location of a user cannot be determined. Thus, theuser may have control over how information is collected about the userand used by the content sharing platform 120.

FIG. 2 is a block diagram illustrating speaker mesh network servercomponent 140 and speaker mesh network client component 115 inaccordance with one implementation of the disclosure. As discussedabove, the speaker mesh network server component 140 may interact with asingle social network, or may be utilized among multiple social networks(e.g., provided as a service of a content sharing platform that isutilized by other third party social networks). In one implementation,the speaker mesh network server component 140 includes a networkmanagement module 210, a playback synchronization module 220, an audiooptimization module 230, and a feedback module 240. More or lesscomponents may be included in the speaker mesh network server component140 without loss of generality. For example, two of the modules may becombined into a single module, or one of the modules may be divided intotwo or more modules. In one implementation, one or more of the modulesmay reside on different computing devices (e.g., different servercomputers, on a single client device, or distributed among multipleclient devices, etc.). Furthermore, one or more of the modules mayreside on different content sharing platforms, third party socialnetworks, and/or external servers.

The speaker mesh network server component 140 is communicatively coupledto the speaker mesh network client component 115 and the data store 106.For example, the speaker mesh network server component 140 may becoupled to speaker mesh network client component 115 and the data store106 via a network (e.g., via network 105 as illustrated in FIG. 1).

In one implementation, the speaker mesh network client component 115includes a device detection module 250, a playback adjustment module260, a sensor reporting module 270, and an audio adjustment module 280.More or less components may be included in the speaker mesh networkclient component 115 without loss of generality. For example, two of themodules may be combined into a single module, or one of the modules maybe divided into two or more modules. In one implementation, one or moreof the modules may reside on different computing devices (e.g.,different server computers, on a single client device, or distributedamong multiple client devices, etc.). Furthermore, one or more of themodules may reside on different content sharing platforms, third partysocial networks, and/or external servers.

The data store 106 may be a memory (e.g., random access memory), acache, a drive (e.g., a hard drive), a flash drive, a database system,or another type of component or device capable of storing data. The datastore 106 may also include multiple storage components (e.g., multipledrives or multiple databases) that may also span multiple computingdevices (e.g., multiple server computers). The data store 106 includescontent item data 290, mesh network data 291, and optimized audio data292.

As discussed above, the speaker mesh network server component 140 andthe speaker mesh network client component 115, individually and/or incombination, enable multiple computing devices, such as mobile devicesincluding phones and tablets, to cooperate as a speaker mesh networksystem in implementations of the disclosure. The audio outputs fromaudio output devices (e.g., speakers) of the computing devices of thespeaker mesh network are synchronized and optimized in order to amplifyaudio output and/or improve the overall audio quality of the audiooutput from the computing devices. As a result, the computing devicesjoined together in the speaker mesh network provide improved audiooutput (e.g., louder and higher quality audio output) of a media itemthan would be provided by a single computing device alone. In oneimplementation, media item may include a video or an audio file, whichmay be stored as content item data 290 in data store 106.

In one implementation, to create a mesh network of computing devices asa combined speaker system for audio, the device detection module 250 ofmesh network speaker client component 115 may include logic to detectother nearby computing devices (e.g., mobile devices) and prompt theother devices to join a speaker mesh network. The device detectionmodule 250 may utilize a variety of processes for speaker mesh networkdevice detection. In one implementation, device detection module 250 mayuse Bluetooth pairing, peer-to-peer WiFi, and/or audio detection ofaudio signatures, for example, to detect other nearby computing devices.

With respect to Bluetooth pairing, the device detection component 250may utilize Bluetooth functionality (provided at the computing devicehosting the device detection component 250) to recognize one or moreother computing devices (in a nearby vicinity defined by the Bluetoothprotocol range) via a Bluetooth pairing protocol. In someimplementations, this may be an automatic process that does not involveuser interaction or it may be an active process requiring affirmativeactions by a user (i.e., opting in, entering a code provided on one ofthe computing devices, etc.).

With respect to peer-to-peer WiFi, the device detection component 250may utilize a WiFi protocol to recognize one or more other computingdevices (in a nearby vicinity defined by the WiFi protocol range) via apeer to peer WiFi protocol. The peer-to-peer WiFi protocol is used toestablish an ad hoc wireless network between computing devices withinthe WiFi protocol range. One of the computing devices may act as awireless router that can take connections from other computing devicesin the area that all connect to this wireless router. Once the computingdevice acting as a wireless network is broadcasting a network, eachcomputing device in the network can utilize the WiFi protocol totransmit data (including audio) between devices.

With respect to audio signatures, the device detection component 250 mayutilize an audio signature to identify nearby computing devices forpurposes of prompting the computing devices to join a speaker meshnetwork. For example, the device detection component 250 may use amicrophone to identify an audio portion of a media item that is playingon another computing device or to listen for an inaudible (to the humanear) signature tone emitted from another computing device. In oneimplementation, the device detection component 250 may listen for anaudio portion that is the same as currently-playing audio of a mediaitem on the computing device. In another implementation, the inaudibleaudio signature tone may identify the application on the other computingdevice that is being used to play a media item.

In some implementations, more than one of the above-described devicedetection processes may be used to detect nearby computing devices andprompt to join a speaker mesh network. For example, the audio signatureprocess may be used to determine that potential computing devices for aspeaker mesh network are nearby, and then the Bluetooth or peer-to-peerWiFi protocol may be used to gather identifying information of thosenearby computing devices for purposes of providing the prompt.

In one implementation, when the device detection module 250 detectsother computing devices using one or more of the above-describedprocesses (e.g., Bluetooth, peer-to-peer WiFi, audio signature), it maycause a prompt to be displayed on its associated computing device toenable the speaker mesh network connection on the computing device. Inone implementation, enabling a speaker mesh network connection on acomputing device may also be referred to as enabling a “multi-devicestereo mode” on the computing device.

FIG. 3A depicts an exemplary screenshot of a UI 300 including a prompt305 according to implementations of the disclosure. Prompt 305 may bedisplayed to notify a user that nearby computing devices have beendetected and to request confirmation from the user for the computingdevice to establish and/or join a speaker mesh network with the detectedcomputing devices. In some implementations, the prompt 305 may provideidentification of the other nearby computing devices (e.g., “Tom'sPhone”, “Caitlin's Phone”, etc.).

FIG. 3B depicts another exemplary screenshot of a UI 310 providing aspeaker mesh network indicator. UI 310 may include a media player havinga playback area 320 displaying playback of a media item, a play button312, a scrub bar 314, a volume indicator 316, and a full screen toggle318. A multi-device stereo mode indicator 322 (e.g., speaker meshnetwork indicator) may be displayed in the playback area 320 whenmulti-device stereo mode is enabled and/or available (i.e., because thecomputing device detects nearby computing devices via theabove-described device detection processes). An add speaker indicator324 may also be displayed to allow a user to manually invite othercomputing devices to join the speaker mesh network.

When a user selects stereo mode indicator 322, UI 330 as shown in FIG.3C, may be provided. FIG. 3C is a block diagram depicting an exemplaryscreenshot of UI 330. UI 330 displays a control 335 to togglemulti-device stereo mode on and off. UI 330 may also include mediaplayer controls, such as play button 312, scrub bar 314, volumeindicator 316, and full screen toggle 318, for example.

Referring back to FIG. 2, once confirmation to join the speaker meshnetwork is received, the device detection component 250 may notify thenetwork management module 210 of speaker mesh network server component140 that the computing device has joined a speaker mesh network. In someimplementations, identifiers of the other computing devices, such as theidentifiers provided in the above-described prompt, are also provided tothe network management module 210. The network management module 210then maintains identifying information corresponding to the speaker meshnetwork, such as a network identifier for the speaker mesh network andindividual identifiers for each computing devices that is a member ofthe network. In one implementation, the identifying information isstored as mesh network data 291 in data store 106.

Once the speaker mesh network is established, speaker mesh networkserver component 140 may act as a master in the speaker mesh network andcoordinate playback, stereo sound, and audio equalization signalingamong the computing devices (e.g., nodes) of the speaker mesh network.The playback synchronization module 220, the audio optimization module230, and the feedback module 240 are utilized to coordinate theplayback, stereo sound, and audio equalization signaling within thespeaker mesh network.

In one implementation, the playback synchronization module 220 may senddirectives to each computing device in the speaker mesh networkregarding which media item (e.g., audio, video, etc.) the computingdevice should play and what playback point of the media item thecomputing device should be synchronized (“synced”) to. In someimplementations, the playback synchronization module 200 assumes thateach computing device maintains an internal clock that is kept in syncwith other computing devices in the speaker mesh network. For thepurposes of establishing an initial playback time, the first computingdevice to join the speaker mesh network may determine the time in themedia item that all other computing devices in the speaker mesh networkare to be initialized to. As a result, the first computing device doesnot experience any interruption in its audio; it simply continues toplay. This first computing device to join may periodically transmit, viaits playback synchronization module 260, its playback status to theplayback synchronization module 220. The playback status may include,but is not limited to, two pieces of information: (i) the timestamp of acurrent playback point in the media item, and (ii) the current readingof the computing device's synchronized clock.

When a second, third, or N^(th) computing device opts to join thespeaker mesh network, the playback synchronization module 220 may send adirective (e.g., instruction) to the joining computing device,instructing the computing device to synchronize its playback to a time[t] in the media item, relative to a synchronized clock time [c]. Theplayback adjustment module 260 on each recipient computing deviceperforms a calculation (e.g., addition) to determine what point in themedia item that the recipient computing device should sync to, relativeto the amount of time that has elapsed since the directive originated onthe first computing device in the speaker mesh network.

For example, the first computing device indicates that playback point isat the 23 s mark when the clock of the first computing device was at6:03 pm and 253 milliseconds. Subsequently, the playback adjustmentmodule 260 of the recipient computing device determines to synchronizeplayback at the 23.5 s mark of the media item due to its clock currentlyindicating the computing device is 500 ms (half a second) later than thefirst computing device's indicated time.

In some implementations, each computing device that is a member of thespeaker mesh network can rewind or fast forward (e.g., scrubbing) in amedia item, and other computing devices in the network can synchronizeto such rewinding and fast forwarding. To achieve this, any computingdevice in the speaker mesh network that advances or rewinds to adifferent playback time in a media item communicates this to the server(e.g., to speaker mesh network server component 140). The playbackadjustment module 260 transmits a time [t] in the media item that thecomputing device scrubbed to, along with a clock reading [c] of when thescrubbing occurred. This information is transmitted to the playbacksynchronization module 220, which then instructs all other computingdevices in the speaker mesh network to synchronize to [t, c]. Uponreceiving this instruction, the playback adjustment module 260 of eachcomputing device calculates its internal time minus the indicated time[c], and moves to the specified timestamp point [t] plus the amount oftime that has elapsed since the directive originated.

In further implementations, the speaker mesh network may be implementedto allow the computing device members to synchronize playback of offlinemedia items when the computing devices have no internet connection. Forexample, the computing devices may synchronize playback ofpre-downloaded audio or video when the computing devices are located ina setting where no internet is available, but peer-to-peer connectionsare possible. In such an implementation, one of the computing devices(e.g., the first one to join and/or establish the speaker mesh network)may act as the master in the speaker mesh network, coordinating playbackand sending directives to the other computing device members of thespeaker mesh network in accordance with the above description. In someimplementations, each client computing device may include a speaker meshnetwork server component 140 that is enabled if no internet connectionis detected. As discussed above, the client computing device that is thefirst to join the speaker mesh network establishes the speaker meshnetwork and acts as the master computing device in the speaker meshnetwork. In some implementations, a user of the client computing deviceauthorizes the client computing device to act as a master computingdevice in the speaker mesh network (e.g., via a prompt provided in a UIby the speaker mesh network client component 115).

In one implementation, speaker mesh network server component 140 mayalso coordinate stereo sound (e.g., two or more tracks of the audioplayed back through different speakers positioned around a listeninguser) and audio equalization signaling among the nodes (computing devicemembers) of the speaker mesh network. The speaker mesh network servercomponent 140 may act as the master for determining where the computingdevice members of the speaker mesh network are positionally locatedrelative to one another, and for determining which stereo sound andaudio equalization settings each computing device should apply.

The sensor reporting module 270 of each computing device in the speakermesh network may periodically transmit one or more sensor readings fromthe computing device to the audio optimization module 230. The sensorreadings may include, but are not limited to, microphone inputs, compassreadings, global positioning system (GPS) readings, and barometricreadings. Utilizing these readings, the audio optimization module 230may determine the overall topology of the different computing devices ofthe speaker mesh network, using the sum of the sensor readings.

For example, based on the strength of sound that is received at themicrophones of a first computing device, the audio optimization module230 can determine where the first computing devices is locatedpositionally relative to another computing device. For example, if thestrength of an audio input is stronger in a left microphone of the firstcomputing device versus a right microphone, the audio optimizationmodule 230 may determine that the first computing device is located tothe right of another computing device. In addition, barometric readings(i.e., altitude) from computing devices may be combined with directionalsensing of audio to determine whether a computing device is locatedabove or below another computing device. Furthermore, compass readingscan provide orientation data for the computing devices, such asindicating whether a computing device is facing forward or backwardrelative to another computing device.

Based on the determined topology of the speaker mesh network, the stereomodule 232 of the audio optimization module 230 may instruct eachcomputing device regarding playback of a particular audio track (e.g.,left track, right track, center audio, etc.) of the media item. A mediaitem may be recorded with two or more tracks (e.g., recorded atdifferent distances through different microphones), and these two ormore tracks may each be played back through different speakerspositioned around a listening user. This may also be referred to asstereo sound. One implementation of stereo sound is right and leftchannel stereo. With right and left channel stereo, a media item isrecorded with two tracks, and the two tracks may be played through twodifferent speakers positioned at the listening user's right and left.The two tracks may be referred to as right channel and left channel oras right track and left track. There also exist systems where a mediaitem may contain multiple audio tracks (more than 2) to allow for“surround sound.” In surround sound, there may be N different audiotracks (e.g., a center track, left track, right track, rear left track,rear right track, and so on).

The stereo module 232 may instruct each computing device, based on theposition of the computing device in the determined topology of thespeaker mesh network, which audio track the computing device shouldplay. Initially, the stereo module 232 may default to having eachcomputing device play the same center track. Then, as sufficient datafrom the computing devices is received to allow the audio optimizationmodule 230 to determine the speaker mesh network topology, the stereomodule 232 may begin to transmit instructions to each computing deviceregarding which audio track/channel of the media item it should play. Inone implementation, the stereo sound track information used to transmitinstructions to the computing devices may be stored as optimized audiodata 292 in data store 106.

In one example, when the media track that is streamed to the computingdevices of the speaker mesh network contains both a left track and aright track, the stereo module 232 may provide one additional piece ofinformation to the client: a number indicating in what proportion thecomputing device should play the left track versus the right track(e.g., 75% right track, 25% left track). If the sensor readings indicatethe multiple computing devices in the speaker mesh network reportnon-contradictory numbers (e.g., three devices are determined to haveleft/right linear positions of −1, +0.1, and +1, respectively), then thestereo mode 232 may instruct the −1 device to play entirely the lefttrack audio, the +0.1 device to play 45% left track audio and 55% righttrack audio, and the +1 device to play entirely the right track audio.

In a more complex surround sound media item, the stereo module 232 maycause different audio tracks of the media item to be streamed torespective computing devices. This avoids using excessive bandwidth,which may occur if the N different audio tracks were all streamed toeach of the different computing devices in the speaker mesh network.

The determined topology of the speaker mesh network may also be utilizedby equalization (EQ) module 234 to coordinate audio equalization of theaudio signal output from each of the computing devices in the speakermesh network. Audio equalization (referred to herein as “EQ”) is theprocess used to alter frequencies in an audio signal. In oneimplementation, EQ may be implemented by applying treble and bassadjustments to an audio signal. The EQ module 234 may utilize the sensorreadings to generate instructions to adjust bass and treble settings ofthe audio signal outputted by the computing devices of the speaker meshnetwork. In one implementation, the audio equalization information usedto transmit instructions to the computing devices may also be stored asoptimized audio data 292 in data store 106.

In some implementations, feedback module 240 utilizes informationgleaned from the overall audio output of the speaker mesh network (e.g.,via periodic sensor readings) and makes adjustments to stereo and EQsignaling to optimize the audio output and audio quality in the speakermesh network. For example, as the computing devices are playingsynchronized audio, the sensor reporting module 270 may continuallyand/or periodically use microphones of the computing devices to listento audio around the computing devices and report the corresponding audioreadings. This information may be utilized by feedback module 240 tocontinually and/or periodically refine and calibrate the audio signalvia instructions from stereo module 232 and EQ module 234.

FIG. 4 is a flow diagram illustrating a method 400 for a mesh network ofcomputing devices as a combined speaker system for audio, according toan implementation of the present disclosure. The method 400 may beperformed by processing logic that comprises hardware (e.g., circuitry,dedicated logic, programmable logic, microcode, etc.), software (e.g.,instructions run on a processing device to perform hardware simulation),or a combination thereof. In one implementation, method 400 may beperformed by speaker mesh network client component 115 as shown in FIG.2.

Method 400 begins at block 402 where one or more nearby computingdevices are detected using a device detection process. As describedabove, the device detection process may include Bluetooth, peer-to-peerWiFi, and/or audio signature analysis, to name a few examples. At block404, a prompt is presented in a UI, where the prompt queries whether tojoin and/or establish a speaker mesh network (e.g., multi-device stereomode) with the one or more detected computing devices. At block 406,confirmation is received (i.e., from a user), via the prompt, to joinand/or establish the speaker mesh network.

At decision block 408, it is determined whether the computing device isto act as a master computing device in the speaker mesh network. In oneimplementation, the master computing device may be a server devicehosting one or more media items that are selectable for playback (e.g.,a content sharing platform). In other implementations, the mastercomputing device may be one of the client computing devices in thespeaker mesh network (e.g., when no internet connection is available).The first computing device to join the speaker mesh network may act asthe master computing device in some implementations. If the clientcomputing device is to act as the master computing device of the speakermesh network at decision block 408, then method 400 proceeds to on-pagereference “A”, which directs to method 500. A client computing deviceacting as a master computing device performs method 500 (described withrespect to FIG. 5 further below) in order to establish the speaker meshnetwork. In some implementations, a user of the client computing deviceauthorizes the client computing device to act as a master computingdevice in the speaker mesh network (e.g., via a prompt provided in a UIby the speaker mesh network client component 115). Subsequent to, and/orconcurrent with, performance of method 500 (e.g., when the clientcomputing device is acting as master computing device), method 400 maythen proceed to block 410. If the computing device is not acting as amaster computing device at decision block 408, then method 400 proceedsto block 410.

At block 410, the received confirmation to join the speaker mesh networkis transmitted to a master computing device. Subsequently, at block 412,sensor readings are obtained and transmitted to the master computingdevice. In one implementation, the sensor readings include microphoneinput, GPS readings, compass readings, barometric sensor readings, andso on. Subsequently, at block 414, playback synchronization instructionsare received from the master computing device regarding a media item tostream as part of the speaker mesh network. The playback synchronizationinstructions may include a time [t] representing a timestamp in themedia item and a synchronized clock time [c]. At block 416, playback ofthe media item is adjusted according to the playback synchronizationinstruction. As discussed above, adjusting playback according to theinstruction may include synchronizing playback of the media item to time[t] relative to the received synchronized clock time [c].

At block 418, stereo sound and audio equalization instructions arereceived from the master computing device. In one implementation, thestereo sound and audio equalization instructions are based on a topologyof the speaker mesh network as determined by the master computing devicebased on the sensor readings received from the computing devices in thespeaker mesh network. At block 420, stereo sound and audio equalizationsettings are applied to the playback of the media item per the receivedinstructions. In one implementation, applying the stereo sound settingsincludes playing indicated proportions of left and right channel tracksof the media item. In another implementation, applying the audioequalization settings includes adjusting bass and treble settings forthe media item audio output.

At block 422, updated sensor readings are transmitted to the mastercomputing device. In one implementation, the updated sensor readings aresent on a periodic basis and result in updated stereo sound and audioequalization instructions being received from the master computingdevice at blocks 418 and 420. If the client computing device performingmethod 400 is acting as the master computing device (per decision block408), then blocks 512 through 518 of method 500 may be repeated toprocess the updated sensor readings and generating updated stereo soundand audio equalization instructions.

FIG. 5 is a flow diagram illustrating another method 500 for a meshnetwork of computing devices as a combined speaker system for audio,according to some implementations of the disclosure. The method 500 maybe performed by processing logic that comprises hardware (e.g.,circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions run on a processing device to performhardware simulation), or a combination thereof. In one implementation,method 500 may be performed by speaker mesh network server component 140as shown in FIG. 2. In some implementations, method 500 may be performedby a client device 110A-110Z as shown in FIG. 1 when such client device110A-110Z is acting as a master client in a speaker mesh network andimplementing the processes of a speaker mesh network server component140 (e.g., when no internet connection is available).

Method 500 begins at block 502 where identifying information isreceived, where the identifying information pertains to one or morecomputing devices and to a speaker mesh network that the one or morecomputing devices have indicated to join. At block 504, the identifyinginformation of the computing devices and corresponding speaker meshnetwork is stored. At block 506, a media item to stream to the speakermesh network is determined. In addition, a first computing device usedto seed the media item playback is also determined. In oneimplementation, the first computing device to join the speaker meshnetwork may be the first computing device used for seeding purposes.

Subsequently, at block 508, a playback timestamp [t] of the media timeand a clock time [c] at the first computing device is determined. In oneimplementation, the first computing device may already be playing backthe media item, and the current timestamp [t] and clock time [c] issampled and/or received from the first computing device. At block 510,playback synchronization instructions are transmitted to the one or morecomputing devices in the speaker mesh network to synchronize playback ofthe media item at time [t] relative to clock time [c].

At block 512, sensor readings from the computing devices of speaker meshnetwork at received. The sensor readings may include microphone inputs,compass readings, GPS readings, and/or barometric readings, to name afew examples. At block 514, a topology of the speaker mesh network isdetermined based on the received sensor readings. At block 516, stereosound and audio equalization instructions are transmitted to thecomputing devices of the speaker mesh network. The stereo sound andaudio equalization instructions are generated based on the determinedtopology of the speaker mesh network, and may differ for each computingdevice in the speaker mesh network.

Lastly, at block 518, updated sensor readings from the computing devicesof the speaker mesh network are received on a periodic basis. Theupdated sensor readings may be used to update the topology of thespeaker mesh network and send updated stereo sound and audioequalization instructions to the computing devices of the speaker meshnetwork at blocks 514 and 516.

FIG. 6 illustrates a diagrammatic representation of a machine in theexemplary form of a computer system 600 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeimplementations, the machine may be connected (e.g., networked) to othermachines in a local area network (LAN), an intranet, an extranet, or theInternet. The machine may operate in the capacity of a server or aclient machine in a client-server network environment, or as a peermachine in a peer-to-peer (or distributed) network environment. Themachine may be a personal computer (PC), a tablet PC, a set-top box(STB), a Personal Digital Assistant (PDA), a cellular telephone, a webappliance, a server, a network router, switch or bridge, or any machinecapable of executing a set of instructions (sequential or otherwise)that specify actions to be taken by that machine. Further, while only asingle machine is illustrated, the term “machine” shall also be taken toinclude any collection of machines that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methodologies discussed herein. In one implementation, computersystem 600 may be representative of a server (such as server 102) orclient (such as client 110A-110Z) executing a speaker mesh networkcomponent 650, such as speaker mesh network server component 140 orspeaker mesh network client component 115, as described with respect toFIGS. 1 and 2.

The exemplary computer system 600 includes a processing device 602, amain memory 604 (e.g., read-only memory (ROM), flash memory, dynamicrandom access memory (DRAM) (such as synchronous DRAM (SDRAM) or RambusDRAM (RDRAM), etc.), a static memory 606 (e.g., flash memory, staticrandom access memory (SRAM), etc.), and a data storage device 618, whichcommunicate with each other via a bus 630. Any of the signals providedover various buses described herein may be time multiplexed with othersignals and provided over one or more common buses. Additionally, theinterconnection between circuit components or blocks may be shown asbuses or as single signal lines. Each of the buses may alternatively beone or more single signal lines and each of the single signal lines mayalternatively be buses.

Processing device 602 represents one or more general-purpose processingdevices such as a microprocessor, central processing unit, or the like.More particularly, the processing device may be complex instruction setcomputing (CISC) microprocessor, reduced instruction set computer (RISC)microprocessor, very long instruction word (VLIW) microprocessor, orprocessor implementing other instruction sets, or processorsimplementing a combination of instruction sets. Processing device 902may also be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. The processing device 602 is configured to executeprocessing logic 626 for performing the operations and steps discussedherein.

The computer system 600 may further include a network interface device608. The computer system 600 also may include a video display unit 610(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), analphanumeric input device 612 (e.g., a keyboard), a cursor controldevice 614 (e.g., a mouse), and a signal generation device 616 (e.g., aspeaker).

The data storage device 618 may include a computer-readable storagemedium 628 (also referred to as a machine-readable storage medium), onwhich is stored one or more set of instructions 622 (e.g., software)embodying any one or more of the methodologies of functions describedherein. The instructions 622 may also reside, completely or at leastpartially, within the main memory 604 and/or within the processingdevice 602 during execution thereof by the computer system 600; the mainmemory 604 and the processing device 602 also constitutingmachine-readable storage media. The instructions 622 may further betransmitted or received over a network 620 via the network interfacedevice 608.

The computer-readable storage medium 628 may also be used to storeinstructions to perform a method for providing a mesh network ofcomputing devices as a combined speaker system for audio, as describedherein. While the computer-readable storage medium 628 is shown in anexemplary implementation to be a single medium, the term“machine-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. A machine-readable medium includes any mechanism forstoring information in a form (e.g., software, processing application)readable by a machine (e.g., a computer). The machine-readable mediummay include, but is not limited to, magnetic storage medium (e.g.,floppy diskette); optical storage medium (e.g., CD-ROM); magneto-opticalstorage medium; read-only memory (ROM); random-access memory (RAM);erasable programmable memory (e.g., EPROM and EEPROM); flash memory; oranother type of medium suitable for storing electronic instructions.

The preceding description sets forth numerous specific details such asexamples of specific systems, components, methods, and so forth, inorder to provide a good understanding of several implementations of thepresent disclosure. It will be apparent to one skilled in the art,however, that at least some implementations of the present disclosuremay be practiced without these specific details. In other instances,well-known components or methods are not described in detail or arepresented in simple block diagram format in order to avoid unnecessarilyobscuring the present disclosure. Thus, the specific details set forthare merely exemplary. Particular implementations may vary from theseexemplary details and still be contemplated to be within the scope ofthe present disclosure.

Reference throughout this specification to “one implementation” or “animplementation” means that a particular feature, structure, orcharacteristic described in connection with the implementation isincluded in at least one implementation. Thus, the appearances of thephrase “in one implementation” or “in an implementation” in variousplaces throughout this specification are not necessarily all referringto the same implementation. In addition, the term “or” is intended tomean an inclusive “or” rather than an exclusive “or.”

Although the operations of the methods herein are shown and described ina particular order, the order of the operations of each method may bealtered so that certain operations may be performed in an inverse orderor so that certain operation may be performed, at least in part,concurrently with other operations. In another implementation,instructions or sub-operations of distinct operations may be in anintermittent and/or alternating manner.

What is claimed is:
 1. A method comprising: receiving, by a processingdevice of a computing device, a prompt to join a speaker mesh network asa master computing device of the speaker mesh network, the speaker meshnetwork comprising one or more other computing devices detected within ageographic vicinity of the computing device; transmitting, responsive tothe prompt, a confirmation to act as the master computing device in thespeaker mesh network; generating, by the computing device acting as themaster computing device, playback synchronization instructions for theone or more other computing devices, the playback synchronizationinstructions to cause playback of a media item to be synchronized at thecomputing device and the one or more other computing devices at aplayback timestamp relative to a clock time; transmitting the generatedplayback synchronization instructions to the one or more other computingdevices; determining a topology of the speaker mesh network based onsensor readings received from the one or more other computing devices;and transmitting stereo sound and audio equalization instructions to theone or more other computing devices to cause stereo settings and audioequalization settings of the one or more other computing devices to beadjusted, the stereo sound and audio equalization instructions based onthe determined topology of the speaker mesh network.
 2. The method ofclaim 1, wherein the playback synchronization instruction comprisesidentification of the media item, a timestamp in the media item, and asynchronized clock time.
 3. The method of claim 1, wherein the stereosound and audio equalization settings at the one or more other computingdevices are different than the stereo sound and audio equalizationsettings at the computing device.
 4. The method of claim 1, wherein theone or more other computing devices join the speaker mesh networksubsequent to receiving a prompt to query whether to join the speakermesh network and providing a confirmation, in response to the prompt tojoin the speaker mesh network.
 5. The method of claim 1, furthercomprising receiving one or more sensor readings from the one or moreother computing devices of the speaker mesh network.
 6. The method ofclaim 5, wherein the one or more sensor readings comprise an audio inputat one or more microphones of the computing device.
 7. The method ofclaim 5, wherein the one or more sensor readings comprise a compassreading of the computing device.
 8. The method of claim 5, wherein theone or more sensor readings comprise a global positing system (GPS)reading of the computing device.
 9. The method of claim 5, wherein theone or more sensor readings comprise a barometric sensor reading of thecomputing device.
 10. The method of claim 5, wherein the one or moresensor readings are utilized to determine the topology of the speakermesh network.
 11. The method of claim 10, further comprising receivingupdated sensor readings from the one or more other computing devices onat periodic time intervals, the updated sensor readings utilized toupdate the topology of the speaker mesh network and to send updatedstereo sound and audio equalization instructions.
 12. The method ofclaim 1, wherein the stereo settings comprise proportions of audiotracks of the media item to output during the playback of the mediaitem, and wherein the audio equalization settings comprise a proportionof bass and a proportion of treble to apply to the playback of the mediaitem.
 13. A computing device comprising: a memory; and a processingdevice coupled to the memory, wherein the processing device is to:receive a prompt to join a speaker mesh network as a master computingdevice of the speaker mesh network, the speaker mesh network comprisingone or more other computing devices detected within a geographicvicinity of the computing device; transmit, responsive to the prompt, aconfirmation to act as the master computing device in the speaker meshnetwork; generate, while acting as the master computing device, playbacksynchronization instructions for the one or more other computingdevices, the playback synchronization instructions to cause playback ofa media item to be synchronized at the computing device and the one ormore other computing devices at a playback timestamp relative to a clocktime; transmit the generated playback synchronization instructions tothe one or more other computing devices; determine a topology of thespeaker mesh network based on sensor readings received from the one ormore other computing devices; and transmit stereo sound and audioequalization instructions to the one or more other computing devices tocause stereo settings and audio equalization settings of the one or moreother computing devices to be adjusted, the stereo sound and audioequalization instructions based on the determined topology of thespeaker mesh network.
 14. The computing device of claim 13, wherein theone or more other computing devices join the speaker mesh networksubsequent to receiving a prompt to query whether to join the speakermesh network and providing a confirmation, in response to the prompt tojoin the speaker mesh network.
 15. The system of claim 13, wherein thesensor readings comprise at least one of audio input at one or moremicrophones of the computing devices, compass readings of the computingdevices, global positing system (GPS) readings of the computing devices,or barometric sensor readings of the computing devices.
 16. The systemof claim 13, wherein the stereo sound and audio equalizationinstructions are different for ones of the one or more other computingdevices.
 17. The system of claim 13, wherein the stereo settingscomprise proportions of audio tracks of the media item to output duringthe playback of the media item, and wherein the audio equalizationsettings comprise a proportion of bass and a proportion of treble toapply to the playback of the media item.
 18. A non-transitorymachine-readable storage medium storing instructions which, whenexecuted, cause a processing device to perform operations comprising:receiving, by the processing device of a computing device, a prompt tojoin a speaker mesh network as a master computing device of the speakermesh network, the speaker mesh network comprising one or more othercomputing devices detected within a geographic vicinity of the computingdevice; transmitting, responsive to the prompt, a confirmation to act asthe master computing device in the speaker mesh network; generating, bythe computing device acting as the master computing device, playbacksynchronization instructions for the one or more other computingdevices, the playback synchronization instructions to cause playback ofa media item to be synchronized at the computing device and the one ormore other computing devices at a playback timestamp relative to a clocktime; transmitting the generated playback synchronization instructionsto the one or more other computing devices; determining a topology ofthe speaker mesh network based on sensor readings received from the oneor more other computing devices; and transmitting stereo sound and audioequalization instructions to the one or more other computing devices tocause stereo settings and audio equalization settings of the one or moreother computing devices to be adjusted, the stereo sound and audioequalization instructions based on the determined topology of thespeaker mesh network.
 19. The non-transitory machine-readable storagemedium of claim 18, wherein the stereo sound and audio equalizationsettings at the one or more other computing devices are different thanthe stereo sound and audio equalization settings at the computingdevice.
 20. The non-transitory machine-readable storage medium of claim18, wherein the playback synchronization instruction comprisesidentification of the media item, a timestamp in the media item, and asynchronized clock time.